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Abstract 


We use a quadratic equation to estimate trends in cohort performances in the charter and 
district sectors on the National Assessment of Educational Progress in 4" and 8" grade between 
2005 and 2017. Data consist of over four million test observations of nationally representative 
samples of students on seven separate math and reading tests. We estimate unadjusted trends as 
well as trends adjusted for changes in the demographic composition of the two sectors. 
Compared to district cohorts, we find steeper upward trends in mean charter performance. 

Larger sector differentials are observed for African Americans, low SES students, and for 
students in the Northeast. No significant trend differences are observed for Hispanic and Asian 
Americans, suburban students and for students in the West. Since students in the two sectors may 
differ in relevant ways for reasons unrelated to demographic characteristics, inferences about 


changes in school production factors remain uncertain. 


Forthcoming in the Journal of School Choice: http://dx.doi.org/10.1080/15582159.2020.1811467 


Introduction 


Public education in the United States is divided into two sectors—school district and 
charter. Schools in both sectors receive most of their funding from governments at the local, 
state, and federal levels. District schools are governed by school boards or other official 
governing bodies, while charters are governed by nonprofit organizations authorized by a public 
agency. The district sector is by far the larger of the two, serving over 94% of all public-school 
students in 2017. But the charter sector grew from roughly 2% of all public enrollments in 2005 
to about 6% in 2017, nearly a three-fold increase (U.S. Department of Education, 2020; Wang, 


Rathbun, & Musu, 2019).! 


After 2016, the rate of growth of the charter sector slackened (Lake et al., 2018). 
Observers have attributed the slowdown to inadequate funding, limited access to appropriate 
facilities, and increased resistance to charter expansion by school districts, teacher unions and 
advocacy groups such as the NAACP (Barnum, 2017; Kahlenberg & Potter, 2014; Katayama, 
2016; Lake et al., 2018; Moe, 2011; (NAACP, 2017). But the rate of growth could also be 
adversely affected if trends in student achievement at charter schools are not keeping pace with 
trends in the district sector. Conversely, the rising political opposition to charters could be a 
function of greater improvement within the charter sector, stoking fears that it could eventually 


disrupt district sector operations. 


To see whether trends in student achievement at charter and district schools differ, we 


track changes in the performance of cohorts of a nationally representative sample of students on 


' We do not discuss the private sector in this paper. The National Assessment of Educational Progress cannot require 
the participation of private schools in its assessments and it therefore does not have a representative sample of 
private schools. 


seven tests in math and seven tests in reading administered by the National Assessment of 


Educational Progress (NAEP) to students in 4th and 8th grade between 2005 and 2017. 


As its name implies, NAEP is a national report card that informs the public on the 
progress state and national cohorts of students are making over time. It is widely used to 
compare student performance across states and to ascertain whether each cohort is learning as 
much as or more than previous ones. The report card also contains information available to 
qualified researchers that can be used to estimate national progress in the district and charter 
sectors with adjustments for demographic characteristics. Our study is the first to make use of 
this information to assess charter and district progress separately. However, NAEP tests students 
on only one occasion, so we are unable to estimate student performance levels using a value- 


added methodology that adjusts for prior levels of student achievement. 


We begin by briefly describing the charter sector and summarizing prior research on 
student achievement in the two sectors. We then describe our data, present our methods, report 


results, and discuss our findings. 


Charters, Districts, and Student Achievement 


The first charter school law was enacted in Minnesota in 1990. The law authorized the 
establishment of a new arrangement for school governance that combined features of the district 
and the private sectors. Forty-three states followed suit, though each state has its own set of 
provisions governing charter operations. To operate, a school must first receive a “charter” from 
a legislatively determined agent: school district, state department of education, mayoral office, 
state university, or other entity specifically given this authority. With important exceptions, 


charters, like district schools, must abide by the provisions of the state legal code, and they are 


funded primarily by government revenues (Batdorff et al., 2015). But charters differ from district 
schools in that they are not operated directly by a government entity such as a school district. 
Instead, charters, like private schools, are operated by nonprofit organizations that have 
autonomy from certain provisions of a state’s legal code. Some nonprofits contract out 
operational responsibility to a for-profit firm. Most choose their own curriculum and personnel 
free of numerous, though not all, state restrictions. Unlike many district schools, most charters 
have not signed collective bargaining agreements with their employees. They are typically 
funded by states (and, sometimes, districts) according to the size and composition of their 


enrollment, with extra funds usually available for students in need of special education. 


Unlike most district schools, charters do not exclusively serve students living within a 
defined geographic area. Instead, they recruit students from the surrounding community 
according to rules that vary from one state to the next. If charters have difficulties persuading 
students to attend, they are at risk financially. If they receive more applicants than they can 
accommodate, they usually must admit applicants at random. However, not every applicant will 
accept admission when they are told the school’s policies and procedures (Pondiscio, 2019). 
Many of the charters that have survived the rigors of building a new school have become quite 
popular with parents, who are more likely to report satisfaction with their child’s school if they 


attend a school in the charter rather than one in the district sector (Cheng & Peterson, 2017). 


But if many charters are popular with parents, it is the conventional wisdom within the 
research community that charters, on average, do no better than district schools at raising student 
achievement. Summarizing an extensive review of the literature, Cohodes (2018, p. 3) says, quite 
frankly, that “the evidence shows, on average, no difference [in achievement] between students 


who attend a charter and those who attend a traditional public school.” There is one exception to 


this generalization, she writes. “Urban charter schools serving minority and low-income students 
that use a no excuses curriculum” have “significant positive impacts on student outcomes (p. 3).” 
She notes that these “no excuses” schools are concentrated in Boston, New York City, and 
Washington, D.C., and perhaps some other urban areas. She also notes that the charters operated 
by the Knowledge Is Power Program (KIPP), a network of schools that follow a “no excuses” 


curriculum, “produces statistically significant positive test score effects (p. 7).” 


In a meta-analysis of 47 charter studies, Betts and Tang (2019) are only slightly more 
positive in their assessment. Apart from students attending KIPP schools and those attending 
middle schools, the authors say, “the predicted gains in achievement from attending a charter 
school are small, typically 0.5 to one percentile point.” But if a student stays in a charter school 
for six to 12 years, Betts and Tang go on to observe, these small gains might become reasonably 
substantial. Their meta-analysis also shows that in middle school gains are somewhat higher, 2 


percentile points in reading and 3 percentile points in math each year. 


In a third review of the literature, Egalite (2020) says that high-quality studies have 
“revealed statistically significant, large, and educationally-meaningful achievement gains for 
lottery winners, with particularly dramatic gains observed for disadvantaged students, students of 
color, and English language learners.” But Egalite also notes that the Center for Research on 
Education Outcomes (CREDO), an organization that has cast its research net widely across many 
states, finds, on average, little difference between charter and district schools. However, CREDO 
shows more positive results for disadvantaged students, those in urban areas, and those in the 


Northeast, with smaller, if any, charter effects in the West (CREDO, 2013; Egalite, 2020). 


Two prior studies that have looked at performance trends in the charter sector. The more 
extensive CREDO (2013) study compares charter students in sixteen states to students at nearby 
district schools who have similar demographic characteristics and prior test performance. The 
study finds relative improvement at charters for each year of the four-year period, 2009 to 2013. 
The study attributes most of the improvement to replacement of weaker charters by more 
effective ones. Baude et al. (2020) compare the value-added performances of students in Texas 
who switch to a charter school with those of students who remain in the original district school. 
The authors found that in the initial period charter school quality was, on average, lower than 
that of district schools. With time, the charter sector improved in relative effectiveness between 
2001 and 2011. The authors attribute that finding to factors such as closure of lower quality 
charter schools, entry of higher-quality charters in subsequent years, and internal improvement 
by charter schools. Changes in the proficiency of students entering the charter sector account for 


a small portion of relative achievement gains. 


In sum, reviews of the charter studies that analyze the two sectors at a single point in time 
do not find much difference between the charter and district sectors, on average, though they find 
considerable variation within the charter sector. The two trend studies find greater progress in 
charters relative to district schools, but the studies are either restricted in their geographic focus 
or track time trends over only a few years. We expand on this literature by describing and 


comparing nationwide trends in the charter and district over a twelve-year period. 


Data 


NAEP is the one reliable source readily available for the study of trends within the 
charter and district sectors. It administers standardized tests in math and reading to samples of 
students that are both nationally representative and representative of students in each of the 50 
states.2 Between 2005 and 2017, NAEP administered seven math and seven reading tests to 
students in the district and charter sectors. We report for the first time a 12-year trend in student 
performances on the seven math and seven reading tests in the charter and district sectors. 
Altogether, over four million tests were administered to district students, and approximately 


140,000 tests were given to charter students (Appendix Table A). 


NAEP’s purpose is to ascertain periodically the levels of performance of cohorts of 
students in math, reading, and other subjects for each state and for the United States as a whole. 
The survey does not identify a specific performance with the name or identity of any student, 
teacher, school, or school district. In fact, no student takes the entire test. Instead, NAEP reports 
plausible values for overall performance on a test estimated from the sample of questions that 
were administered to a student. There is thus no incentive for a school or a district to excessively 


prepare their students for the test in order to maximize performances. 


Despite the large number of test observations, NAEP does not conduct a census of 
student performance, which would require that all students be tested. Instead, it draws 
representative samples of schools and representative samples of 4th- and 8th-grade students 
within each of the sampled schools. If a school is unable or unwilling to participate, NAEP 


substitutes another school with a similar set of student backgrounds and other characteristics. If 


> Our data come from what is called the Main NAEFP to distinguish it from the Long-Term Trend (LTT) NAEP, 
which tracks achievement levels of nationally representative samples of student cohorts. Samples collected for the 
LTT NAEP are much smaller, and information on whether the student is attending a charter school is only available 
for 2008 and 2012. We refer to the “Main NAEP” simply by its acronym, NAEP. 
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the randomly chosen student at a school is unable or unwilling to participate, a comparable 
student is tested instead. The data are weighted to reflect the composition of the student 


population within each state. 


The NAEP lens for viewing student achievement, though better than any other available 
for this purpose, is not the equivalent of a Hubble telescope. For one thing, we cannot be sure 
that the sample of charter students is nationally representative. Rather, the sample is drawn and 
weighted to be representative of the total 4th- and 8th-grade public-school population for each 
state and for the United States as a whole, not to be representative of each sector. To ascertain 
the representativeness of NAEP observations of charter school performance, we compare the 
characteristics of the NAEP sample to the characteristics of the census of schools and students 
conducted by the National Center for Education Statistics (NCES), the other authoritative source 
for education statistics in the United States. Instead of the sampling procedure used by NAEP, 
NCES conducts an annual census by asking states and local districts to respond to detailed 
questionnaires that ask about enrollments, finances, school personnel, and other factors. NCES 
has gradually introduced procedures that allow it to collect information about schools and 


students in the charter sector. 


NCES, too, is a less-than-perfect data-collection instrument. The information NCES 
obtains is only as accurate as are the responses to its survey, which may include ambiguous 
questions open to alternative interpretations. For example, at one time it was not clear whether 
charter enrollments should be counted as part of district enrollments, whether they should be 
separately identified, or whether each charter school should be treated as a separate district. The 
variation in the legal framework for charters across states added to the confusion. Even now, the 


number of charter schools reported by NCES differs from the number reported by state agencies, 


mainly because the definition of a school (as distinct from a campus that is part of another 
school) is not consistently applied. However, NCES figures for the number of students enrolled 


in charter schools are not seriously disputed. 


Even if both data-collection systems were ideal, one would still expect to see differences 
between them. For one thing, NCES gathers information on enrollment as of October 15, while 
NAFP tests students later in the school year. Students may change schools in the meantime. 
More important, NCES obtains information on all enrolled students, while NCES administers its 
tests only to a representative sample of students in 4th and 8th grade. Unless the composition of 
students in these grades is identical to overall school enrollment at each level, the NCES and 
NAEP estimates of student characteristics will differ. That said, one still expects the NCES and 


the NAEP data to show similar trends and to be generally consistent with one another. 


The overall trends in the NCES and NAEP data are broadly comparable (Table 1). The 
NCES reports that in 2006 the charter share was 2% of the elementary public-school population; 
by 2017, the share had expanded to 6%. The charter share of observations in the NAEP math and 
reading tests shows a similar trend. In 2005, the share of both 4th-grade and 8th-grade students 
was 1.5%.> By 2017, the percentage of 4th-grade students taking the tests had increased to 3.9% 
and the percentage of 8th-grade students had increased to 5.6%, somewhat higher than in the 


census of elementary schools.. 


NCES and NAEP also provide information on the ethnic composition of the charter and 


district sectors.t NCES shows a decline in the White share of the charter sector from 43% in 


3 For NAEP, we use the weighted number of observations. 
4 Both have information on participation in the free and reduced-price lunch program, but the data are reported in 
such different ways they preclude comparisons. 


2001 to 36% in 2011 to 32% in 2018 (Table 1). The percentage of White 4th-grade students 
participating in NAEP tests shows a similar downward trend—from 42% in 2005 to 36% in 2011 
to 34% in 2017. The White share of tested 8th-grade students declines from 39% in 2005 to 31% 


in 2011 and to 30% in 2017. 


NCES reports that the Black share of charter-school elementary enrollment also declines 
from 33% in the spring of 2001 to 29% in 2011 to 26% in 2018. The Black share of tested 8th- 
grade students is higher, but it also falls from 37% in 2005 to 32% in 2011 and to 29% in 2017. 
The Black share of 4th graders in the NAEP testing data is also higher, but it trends up and down 


over time—from 34% in 2005 to 37% in 2011, then down to 32% in 2017. 


NCES reports the Hispanic American share as climbing from 19% in 2001 to 27% in 
2011 to 33% in 2018. The Hispanic American share of those tested by NAEP in 4th grade is 
lower, but it also increases from 19% to 21% between 2005 and 2011 and then to 26% in 2017. 
Among 8th graders, the Hispanic American share increases from 19% in 2005 to 31% in 2011, 


where it remains in 2017.° 


The Asian American share in NCES was 3% in 2001, 3.7% in 2011, and it rose to 4.4% 
in 2018.° The Asian American share of those tested in 8th grade by NAEP climbs steadily 
upward from 2.8% to 6%. The trends for 4th-grade students show a minor increase from 3.2% to 


4.4%. 


In sum, both NCES and NAEP data indicate that the charter sector is predominantly non- 
White, and both show that the non-White share of the total has increased over the course of the 


21st century. However, NAEP administers its math and reading tests to a higher percentage of 


5 Since we have no information on the citizenship of those tested, we refer to all participating students as Americans. 
® We use Asian to identify Asian/Pacific Islander categories in NAEP and NCES. 
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Black students and a lower percentage of White American and Hispanic American students than 


NCES reports. 


Discrepancies could be a function of a higher share of ethnic differences of students 
attending charters in 4th and 8th grade than in other grades. Or it could be due to differential 
procedures used to classify students into ethnic groups, as procedures for classifying students 
into Black or Hispanic or other categories may differ by district and are open to alternative 
interpretations. Or it could be that White Americans and Hispanic Americans are under-sampled 
in the NAEP achievement tests. Given these inconsistencies, it is important to look at trends 


within ethnic groups as well as the overall trend. 


Methodology 


Like NAEP, we focus on changes in cohorts of students from one assessment to the next 
in order to get a sense of the direction in which a sector is heading. In that respect, our study 
differs from earlier research that compares the relative effectiveness of the charter and district 
sectors at raising the achievement of individual students in specific places at particular points in 
time. That is not our purpose here. We do not observe the progress of any one student but rather 
the changes in performance of one student cohort to the next. We look at these cohort changes in 
both the district and charter sectors in order to see the trends occurring in each sector and to 


compare those trends to one another. 


Our analysis, like other research on the two sectors, must attend to what is known as the 
selection problem, the fact that students choose whether to attend schools in the charter or district 
sector. Since each sector is self-selected, it is not easy to make apples-to-apples comparisons 


between them. If one sector attracts more proficient students than the other, differences in sector 
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performance may be due to differential recruitment rather than to any changes in the schools 
themselves. For example, the charter sector, as it grows, might be attracting an increasingly 
advantaged group of students. Or, conversely, charters may be attracting more students who are 
educationally challenged. Many charter schools admit students by a lottery, so it might be 
thought that there is no opportunity for the charter sector to become more or less selective over 
time. But lotteries take place only among those who apply to charter schools, and not every 
student who applies accepts admission. And some charter schools do not receive enough 


applicants to require a lottery. 


Similarly, the demographic composition of the district sector can change both as the 
result of choices students are making and as the result of demographic change in society. For 
example, the Hispanic American and Asian American shares of the school-age population 
increased between 2005 and 2017, while the share of White Americans attending district schools 


declined. These demographic changes may affect performance levels. 


We correct for such changes in the composition of the two sectors by adjusting or 
controlling for background characteristics known to be associated with student achievement, 
including a child’s socioeconomic status (SES), ethnicity, and gender. We use such adjustments 
as a tool to ascertain how much progress in each sector may be due to demographic change as 
distinct from changes in school factors. However, we are unable to control for student test 
performances in previous years and therefore cannot employ a value-added model that has often 
been used to estimate charter effectiveness (CREDO, 2013.) We therefore cannot detect ways in 
which the charter sector might become increasingly selective other than by observing changes in 


the demographic characteristics of the two sectors. Our comparisons, while suggestive, cannot be 
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interpreted as definitive evidence that trends observed for either sector are due in part to 


heightened school quality. 


We first report the mean test performance of initial 4th- and 8th-grade cohorts in reading 
and math in 2005. To equate results across tests, we use the standard deviation (sd) of these 
cohorts to calculate achievement means in sd for all NAEP math and reading assessments at 
grade 4 and grade 8 that are administered between 2005 and 2017.’ We estimate trends in mean 
performances over time by calculating the distance (in sd) of the test-score distribution for each 
cohort’s performances in each subject and grade level from the means of the initial observations 
in 2005, which are set to zero. The empirical models are analyzed separately for each grade and 
subject. In all estimations, we apply the weights NAEP uses to make samples state and nationally 
representative. Following Reardon (2011) and Hanushek et al. (2020), trends have been 
estimated with a quadratic function to allow for estimates of nonlinear change. We extract the 
performance trend of cohorts in math and reading for each year of test administered at each grade 


level with the following equations: 


Osgi = %o + yt + A2t? + Esgti, Where (1) 


O is the achievement score for student i, by subject s, testing grade g, and cohort t; The 


parameters a, and az describe the achievement trend, and ¢ is the error term. 
When controls X are added the equation becomes 


OSgi =A + ayt + Ste + BY Xi + Esgti (2) 


T As mentioned, NAEP estimates plausible values of overall test performance of individual students from their 
performance on the section administered to them. We use the second plausible value NAEP provides. Results are 
robust to estimating effects with averages of the first five plausible values. 
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where X includes dummies for ethnicity, gender, free and reduced-price lunch status at grades 4 
and 8. For 8th-grade students, levels of parental education, computer availability in the home, 
and the number of books in the home are used as additional controls for student’s socioeconomic 
status (SES). We divide the student into four quartiles of the SES distribution that is estimated by 
extracting the first principal component from a factor analysis based upon these same variables 
(Broer, Bai, & Fonseca, 2019; OECD, 2018).° We do not include special education or English 
Language Learner status as controls, because the two sectors may have different definitions of 
these concepts (Ackerman & Egalite, 2017; Setren, 2015; Winters, Carpenter, & Clayton, 2017). 
We use the fitted point estimate and standard errors of the start and end points for each model to 
estimate the overall changes and associated standard error (Feiveson, 1999; Gould, 1996; 


Oehlert, 1992; Philips & Park, 1988).° 


All differences between student performance at charter and district schools for each 
subject at each grade level are assumed to be statistically significant if the differences in the 
point estimates for each trend exceed the joint standard errors of the two estimates. Other than to 
indicate a null effect, we do not interpret in the text differences that are not statistically 


significant. 


Because the NAFP tests are linked by subsets of questions asked of students at both 4th 
and 8th grade, one can estimate the difference in standard deviations between the average 


performances of 4th- and 8th-grade students. The estimated average difference between 4th- and 


8 In the factor analysis, two variables are included. The first is parent education, which is based upon student reports 
as to whether the parent with the highest level of educational attainment 1) did not finish high school; 2) graduated 
from high school; 3) received education beyond high school; or 4) obtained a college degree. The second variable is 
an index that sums two variables: 1) whether or not a computer is in the home; and 2) a measure of the rank ordering 
of students’ estimates of the number of books in their home (divided into the following four categories: 0-10, 11— 
25, 26-100, and >100). A separate factor analysis is performed for each assessment. 

° The calculations are based on the delta method. This method calculates the variance, standard error, and Wald test 
statistic (z-test) from the nonlinear transformations of the estimated parameter vector from the fitted model. 
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8th-grade students on NAEP math and reading tests administered in 2005 and 2017 is 1.23 sd. 
Dividing that number by four yields 0.31 sd, the average amount of learning that takes place each 
year between grades 4 and 8. We use this metric to interpret observed differences in sd in years 
of learning. For example, the cohort gain of 8th grade in math for students in charter schools 


between 2005 and 2017 is 0.34 sd or slightly more than one year’s worth of learning. 


Because tests at 4th and 8th grade are designed to be on the same scale, we report 
averages for 4th- and 8th-grade math and reading tests combined, and many of the findings 
discussed in the text are for the average of the four tests given in the two subjects at both grade 
levels. We present the information in this way in order to reduce the number of specific results 
reported in the text to a manageable number. However, these averages mask a number of 
differences by both test and grade level. When adjusting for background characteristics, we 
control for parent education and home possessions only in the 8th-grade analyses, because NAEP 
does not ask 4th-grade students these questions. For more precise understanding of trends over 
time, the reader is encouraged to examine results more precisely in the tables that present the full 


set of results by subject and grade level. 


Results 


We first report average cohort performance levels on NAEP’s math and reading 
assessments in 2005 for both the charter and district sectors. We next show the changes in the 
mean performance on these tests for all students in each sector both with and without controls for 
background characteristics. We then examine trends by ethnicity, SES, eligibility for free and 


reduced-price lunch, region, and locale. 


NAEP achievement levels in 2005 and 2017 
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On average, district schools outperformed charter schools in 2005. In math, the average 
score for the cohort in 4th grade in district schools was 237 points (Table 2), 5 points or 0.15 
standard deviations (sd) higher than the charter average. In reading, the charters trailed districts 
by 2 points. The district advantage in 8th-grade math in 2005 was 10 points or about 0.28 sd, and 
in reading it was 5 points. By 2017, most of these differences disappeared, or nearly so. At grade 
4, charters still trailed districts by 3 points in math, but in reading they edged above districts by a 
point. At the 8th-grade level, charters also slipped ahead by a point in reading and secured a tie 


with districts, on average, in math. 


Trends in mean cohort performance 


We also observe a disproportionately positive trend for students at charter schools, as 
compared to those at district schools, when we estimate trends with information on performances 
on all seven of the NAEP math tests and all seven of the reading tests. As shown in Figure 1, the 
district trend in 8th-grade math rises modestly in the early portion of the period, only to recede 
more recently. In reading, the district trend rises, then flattens in the second half of the period. 
Charter gains in reading are quite linear and rise more steeply than those of students at district 
schools. The math performances of cohorts at charters rise even more steeply at first but flatten 
toward the end. It is difficult to interpret these uneven, disparate fluctuations, so in the analysis 


that follows we focus on the estimated change for the period overall. 


The average performance of cohorts of students in the district sector improves modestly 
over the 2005-17 period. As can be seen in Table 3, the average combined gains in student 
performances in math and reading at grades 4 and 8 is 0.1 sd. The achievement of students at 


charter schools improves at a more rapid rate. Overall, the average combined gains for the two 
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subjects and grade levels is estimated to be 0.24 sd or 0.14 sd greater than the average district 
gain. Relative gains at charters, as compared to districts, are larger in 8th-grade math (0.24 sd) 
than in 8th-grade reading (0.11 sd). In 4th grade, the relative gains are 0.1 sd and 0.12 sd for the 
two subjects, respectively. When background characteristics are controlled, the average student 
gains for both grades at both levels are estimated to be 0.21 sd at district schools and 0.30 sd at 
charters, a difference of 0.09 sd or roughly two thirds the size of the uncontrolled estimate (Table 
3).!° That difference is somewhat larger at the 8th-grade level than at the 4th-grade level, a 
finding consistent with research that reports larger charter impacts in middle schools (Betts & 


Tang, 2019). 


In sum, the upward trend appears to be steeper in the charter sector than in the district 
one. Roughly one third of the charter gain relative to district schools is attributable to changes in 
the observed characteristics of the district and charter populations, a suggestion that the charter 
sector is recruiting a more proficient set of students over time. The remaining two thirds of 
charters’ relative gain could be due to differential changes in school production factors in the two 
sectors, though unobserved differences in student characteristics cannot be ruled out as an 


alternative explanation. 


Ethnicity 


'OWe conduct two robustness checks, which confirm our main analysis. First, we estimate mean effects without 
controlling for FRL. Results show relative gains of charter (compared to district) cohorts of 0.05 sd and 0.08 sd in 
4th-grade math and reading, respectively. For 8th-grade cohorts, the numbers are 0.20 sd and 0.08 sd. Second, we 
estimate effects with controls for students identified as participating in special education and limited English 
proficiency programs. With these added controls, relative charter gains are 0.06 sd and 0.09 sd in 4th-grade math 
and reading, respectively. In 8th grade the relative gains are 0.16 sd and 0.07 sd. 
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To see whether cohort gains by districts and charters vary by ethnicity, we estimate 
changes in cohort achievement for four broad ethnic categories—African American, White 


American, Hispanic American, and Asian American. 


African American. In 2005, the average performance of all African Americans who took 


the 4th-grade math test was the lowest of the four ethnic groups, over 0.93 sd less than the 
average performances of White Americans and 1.11 sd less than Asian Americans, differences 
similar to those reported elsewhere (Jencks & Phillips, 1998; Reardon, 2011; Magnuson & 
Waldfogel, 2008). In 8th grade, the Black-White disparity in math was even larger—1.2 sd. The 
Black-Asian gap was just as dramatic at the 8th-grade as at the 4th-grade level. In reading, these 
ethnic gaps were somewhat less—approximately 0.80 standard deviations. It was the size of 
these gaps that persuaded many states to establish charter schools as an alternative to traditional 
offerings in the district sector. Perhaps this is one reason African Americans remain the largest 
ethnic group attending schools in the charter sector. They constitute about 33% of the tested 


charter school populations between 2005 and 2017. 


African Americans are also the ethnic group that shows the steepest increase in average 
cohort performance at charters, relative to the district sector between 2005 and 2017 (Table 4). In 
the district sector, the combined average gain by African Americans in both reading and math at 
both 4th- and 8th-grade levels is 0.14 sd. The gain is roughly the same at both grade levels. In the 
charter sector, the combined average gains by African American students are more than twice as 
large—0.33sd. The charter cohort gain in 8th-grade math is a very sizable 0.46 sd, four times 
larger than in the district sector. In reading it is 0.33 sd. At the 4th-grade level, the average gain 


for the two subjects is about 0.33 sd, nearly twice the gain in the district sector. Altogether, the 
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relative gains of the charter sector for the two grades and two subjects combined amount to 0.19 


sd or about half a year’s worth of additional learning for African American students. 


These results do not necessarily show that the quality of schools attended by African 
Americans in the charter sector are improving at a more rapid rate than those in the district 
sector. It may be the case that African American students who attend charters are increasingly 
coming from backgrounds that leave them better prepared for school. To see whether there is 
observable evidence in our data set for such a shift, we control for family background 
characteristics. When this is done, we find a 0.17 sd relative improvement among cohorts of 
African American charter students (as compared to district ones), almost as large as gains of 0.19 
sd observed in the uncontrolled estimations (Table 4). In other words, almost none of the gains in 
test-score performance can be explained by changes in the background characteristics of African 
American students attending charters. It is quite likely that charters induce a half year’s greater 
improvement relative to district schools in African American achievement over this period, 
though unobserved changes in background characteristics might contribute to this result. Still, 
the trends observed here are consistent with studies that show sizable charter impacts on the 
performance of African American students. The average gains for this ethnic group appear to be 
generalizable beyond Boston, New York City, and Newark, where experimental studies of 
relative effectiveness have been undertaken (Cohodes et al, 2013; Dobbie & Fryer, 2013; Hoxby, 


Muraka, & Kang, 2009; Winters, 2020). 


White Americans. White American students account for 35% of the 4th-grade and 32% 


of the 8th-grade charter school population tested by NAEP over the 12-year period. Steeper gains 
for cohorts of students at charter schools, relative to those at district schools, are detected (Table 


4). 
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In district schools, the combined average gain in math and reading in 4th and 8th grade 
between 2005 and 2017 by cohorts of White students comes to 0.1 sd. When adjusted for 
changes in background characteristics, the change is 0.18 sd. The unadjusted combined average 
gains over the same period in the same grades and subjects by cohorts of Whites in the charter 
sector come to 0.22 sd, over twice the 0.10 sd gains observed in the district sector. When 
adjusted for background characteristics, these gains remain intact (0.24 sd). Relative to the 
district sector, charter improvement among White Americans in 4th and 8th grade in both 
subjects combined amounts to 0.12 sd. Half of that gain (0.06 sd) persists after controlling for 
student background. From this it may be inferred that the differential progress by cohorts of 
White students at charters is due almost equally to demographic changes and to school 
production factors, assuming controls for background characteristics capture all selection effects. 
Note that differential changes that might be attributed to school production factors cannot 
account for any more than 0.06 standard deviations of the gain for White Americans, little more 


than a third the size for African Americans (0.17 sd.) 


In sum, our findings are consistent with prior research showing fewer achievement 
benefits in the charter sector for White Americans than for African Americans (CREDO, 2013; 
Cohodes, 2018). We nonetheless see evidence of improvement in relative school quality within 


the charter sector for White Americans. 


Hispanic Americans. Hispanic American students comprise 24% of charter students 


tested in 4th grade and 30% of those tested in 8th grade. In contrast to the results for Black 
students and White students, we find no clear difference in the trends in performance in the 
district and charter sectors between 2005 and 2017 (Table 4). The average combined 


performance of Hispanic students in both 4th and 8th grade rose by 0.21sd in both sectors. These 
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strong gains persist after controlling for background characteristics, emerging as one of the most 


notable aspects of U.S. education over this time period. 


Asian Americans. Much the same can be said for Asian Americans, the smallest ethnic 


group within the charter sector. They comprised only 4% of the 4th-grade charter students and 
5% of the 8th-grade charter students. The advances in performance are higher for this segment of 
all tested students in both district and charter sector than for the three larger groups (Table 4). 
Average gains are about a third of a standard deviation, or roughly a year’s worth of learning, in 
both district and charter sectors. Those gains remain essentially unchanged after controlling for 
background characteristics. There is little difference in changes in performances between the 


charter and district sectors. 


Summary. In sum, the test performances of cohorts of African American and White 
American students in charters advanced at a steeper rate than cohorts of students in district 
schools between 2005 and 2017. Among African American students, the cohort gains in charter 
schools appear to be due mainly to changes in school production factors, while the gains 
registered by White cohorts appear to be due to both demographic change and school factors. 
Hispanic Americans are advancing steadily in both sectors, and Asian Americans are leading the 


way in both sectors, with little difference between them. 


Socioeconomic status 


Due to data limitations discussed above, we estimate trends in cohort performance by 
quartiles of the SES distribution only for students in 8th grade. The performance of cohorts of 


district students in the top SES quartile trends upward by 0.02 sd. in math and 0.09 sd in reading. 
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(Table 5). When other background characteristics are controlled, the shift upward is slightly 
steeper—0.4 and 0.13 in the two subjects, respectively. The performance of cohorts of students 
in the top SES quartile who attend charters climbs at the more rapid rate of 0.27 sd in math and 
0.21 sd in reading over the period. The magnitude of the trend at charters remains unchanged 
when adjustments are made for background characteristics (0.21 sd and 0.22 sd, respectively). 
Taking the two-subject average, cohorts of charter 8th-grade students in the highest SES quartile 


are showing a greater gain of 0.13 sd than cohorts of district students. 


The differentials between the two sectors are greater for 8th-grade students in the bottom 
quartile of the SES distribution. In the district sector, the students in the lowest SES quartile 
climb upward by 0.21 sd in math and 0.24 sd in reading, a larger gain than for the top quartile, 
which suggests that a modest closing of the SES achievement gap is taking place. At charters, the 
vertical tilt among the lowest quartile is more precipitous—0.48 sd in math and 0.31 in reading. 
Estimates are not altered materially when background controls are introduced. When the two 
subjects are combined, the differential in the trends between the two sectors are 0.17 sd., 
approximately one-half year’s worth of learning. The results are consistent with prior research 


that shows more positive charter effects for less advantaged students (CREDO, 2013). 


Income (eligibility for free or reduced-price lunch) 


Charter students are more likely to qualify for free and reduced-price lunch status than 
their counterparts at district schools (U.S. Department of Education, 2018). In the analysis of 


trends in average cohort performance, we control for student eligibility for participation in the 
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free or reduced-price lunch program (FRL), as eligibility depends mainly on a student’s 
household income. But the indicator is not without its hazards, especially for trend analyses 
(Chingos, 2016; Domina et al., 2018). Between 2005 and 2017, eligibility for the subsidy was 
broadened to include those with higher incomes, and, in recent years, all students in some school 
districts with a high incidence of poverty were declared eligible. As a result, changes in student 
performance for both those eligible and ineligible for FRL are not consistent over time. Instead, 
those not eligible for the government subsidy are steadily becoming a more income-exclusive 
segment of the population. Likewise, a broader income group is becoming eligible for assistance 
as time goes by. Because our findings may be contaminated by changes in eligibility rules, we 


place less weight on these results than on the others we report. 


Among students from the higher income (FRL ineligible) families, cohorts in district 
schools performed 0.24 sd higher in 2017 than in 2005 on the combined reading and math tests 
administered in 4th and 8th grade (Table 6). In charters, the gain was 0.35 sd. over this period, a 
modest 0.11 difference. Among those of lower income who were eligible for FRL, the average 


gain was 0.16 sd in districts and 0.31 sd in charters, a difference of 0.15 sd. 


Given the changing definitions of eligibility for participation in the FRL program over 


this period, these results should be interpreted cautiously. 


Region 


To see whether charter improvement rates vary by regions of the country, we divide the 
United States into four regions, as defined by the U.S. Census—Northeast, Midwest, South, and 


West. We find the largest cohort improvements at charters, relative to district schools, in the 


pe 


Northeast, somewhat less differential cohort gains at charters in the Midwest and the South, and 


little or no difference in the cohort gains at district and charter schools in the West. 


Northeast. In the northeastern region, cohorts of students in district schools improved, on 
average, by only 0.05 sd over the period on the combined math and reading tests at the two grade 
levels (Table 7). At charters, cohorts scored, on average, 0.19 sd higher in 2017 than in 2005 on 


the two tests at the two grade levels. 


The differences are considerably larger when changes in student background are taken 
into account. When that adjustment is made, performance improvement by charter cohorts in the 
Northeast comes to 0.38 sd, a gain of over a year’s worth of learning. That is considerably larger 
than the 0.13 sd higher rate of improvement evident in district schools of the Northeast when 
estimates are adjusted for background characteristics. The difference between the two sectors 
amounts to 0.24 sd or about two thirds of a year’s worth of learning. The adjusted relative gains 


at charter schools are larger for 8th-grade students than for the 4th-grade students. 


Midwest. In the Midwest, combined average gains in performance of cohorts of charter 
students in 4th and 8th grade is estimated to be 0.25 sd, as compared to 0.08 sd in the district 
sector (Table 7). After adjusting for changes in background characteristics, charter gains remain 
essentially unaltered (0.28 sd), suggesting that most of the charter gains are likely due to elevated 
school quality, not to the selectivity with which students are recruited to the charters. District 
schools register a gain of 0.17 sd once adjustments are made for changes in student background 


characteristics. 


South. The pattern of results in the South resembles that in the Midwest. The combined 


performance of charter students in 4th- and 8th-grade math and reading improve by an average 
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of 0.19 sd. When adjusted for background characteristics, the estimate of student performance 
improvement (0.25) is somewhat higher, an indication that the gains may be due to enhanced 
school quality. However, there are only modest charter gains of 0.07 relative to district schools in 
the South, which show an improvement of 0.19 sd in both subjects at both grade levels after 
adjustments for background characteristics. In other words, the district schools in the South show 
larger gains than in either the Northeast or Midwest, but southern charters nonetheless improve 


at a marginally higher rate. 


West. When statistically adjusted for background characteristics, district schools in the 
West show average gains of 0.28 in student performance in math and reading at both grade 
levels, higher than in any other region of the country. When the same statistical adjustments are 
made for students attending charter schools, the gains in the West are 0.25, just short of those 


registered in the district sector. 


Although our findings are tracking trends over time, they are broadly consistent with 
studies of charters at single points in time, which also show the most clearly positive impacts of 
charters on students living in the Northeast (Cohodes et al, 2013; Dobbie & Fryer, 2013; Hoxby, 
Muraka, & Kang, 2009; Winters, 2020). Other studies show neutral to negative results in the 


West (CREDO, 2009; 2014). 


Locale 


NAEP allows for the analysis of changes in student performance by the type of 
community in which a school is located. Since the locale variable in NAEP is consistent with 


U.S. Bureau of the Census definitions of localities only since 2007, we track these trends over a 
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10-year rather than a 12-year period. Cities are defined as the principal city within any urbanized 
area. Suburban areas are those communities outside the principal city within an urbanized area. 
Small towns and rural areas do not lie within an urbanized area or an urban cluster. We find 
larger relative gains for students in charters located in cities than for those in suburban charters 


(Table 8). 


Cities. Charters are primarily an urban phenomenon. Sixty-five percent of charter 
students tested by NAEP attend schools located in cities at the core of urban areas. Between 
2007 and 2017, the combined improvement in both subjects and at both grade levels of students 
attending urban schools in the charter sector, relative to those in the district sector, is, on average, 
0.15 sd. When background characteristics are controlled, relative charter gains are just 0.08 sd, 
which may imply that half the gains are due to greater selectivity in the charter sector and half to 


elevated school quality. 


Suburbs. Twenty percent of the tested students live in suburbs, communities outside the 
core of urban areas. In these suburbs, combined relative gains at charters over the period were 
0.08 sd in both subjects and grade levels, but once background characteristics are controlled, no 
differences are observed when subjects and grade levels are combined. But there is a sharp 
contrast by grade level in suburbia. The upward trend for charter students is less than the one for 
district students at the 4th-grade level, but the opposite is true at the 8th-grade level when 
demographic controls are introduced. This result may be due to the better controls for SES 
available for 8th-grade students. But it may also be due to differential school quality in the two 


sectors at the two grade levels. 


Towns and rural areas. Only 12% of the students tested by NAEP live in towns and rural 


areas outside urbanized areas. While public opinion polls show more limited support for charter 


26 


in rural areas than in urban areas (Shakeel & Henderson, 2019), the scholarly data on student 
performance at rural charter schools are thin. Given the limited number of NAEP observations 


for student performance at charters in towns and rural areas, we cannot fill in this gap. 


Summary. The inconsistent results for students at suburban schools resemble the neutral 
to negative findings reported by impact research (CREDO, 2013; Gleason et al., 2010). 
However, the trends for charters in urban areas are less than might be expected, given the charter 
impacts for urban charters in some prior research (Abdulkadiroglu et al., 2017; Angrist et al., 


2013). 


Discussion 


In this, the first comparative assessment of changes in student achievement of cohorts of 
students within the charter and district school sectors between 2005 and 2017, we find that in 
both sectors average cohort performances in math and reading in 4th and 8th grade improve over 
the period. However, the gains by cohorts of charter students are approximately twice as large as 
those registered by cohorts in the district sector. About one third of this difference can be 
explained by changes in the background characteristics of students within each sector, a signal 
perhaps that charters strengthened their capacity to recruit more proficient students. The other 
two thirds of the difference in trend lines for the two sectors may tentatively be attributed to 
enhanced charter performance relative to that of district schools if one assumes that changes in 
student composition are fully captured by changes in demographic characteristics. Our finding is 
consistent with a Texas study and a 16-state CREDO study, both of which identify relative 
school productivity gains in the charter sector, which are attributed largely to replacement of 


weaker charters with more productive ones (Baude et al., 2020; CREDO, 2013). 


Pua 


Our study is limited in several ways. Although we adjust for student demographic 
characteristics when estimating sector effects, we are unable to control for prior student 
achievement. Further, we are unable to identify the moderating impacts of specific factors on 
trends in the charter sector, such as school size, teacher-pupil ratio, per pupil expenditure, years 
of operation, type of charter authorizer, charter sector experience within a state, share of charters 
in networks (rather than a standalone operation) or a host of other potential moderating variables. 


Those worthy topics we leave for further research. 


We nonetheless conclude that the rate of relative improvement in the charter sector is 
likely due both to increasing selectivity and to enhanced charter performance is consistent with 
expectations. A sector would not become increasingly attractive to a better-prepared clientele if it 
was not improving its offerings. Consider transistor radios and television sets, the classic 
examples of a disruptive innovation (Christensen et al., 2011). Initially, the product was of low 
quality, and the clientele consisted largely of those who perceived no reasonable alternative. As 
product developers enhanced the devices, their market share increased and broadened to include 
more prosperous customers, eventually driving the makers of the vacuum tube devices out of 
business. The dynamics of the education market can be expected to take much longer than the 
transistor market, since it is a less competitive market. But selectivity and quality are likely to 
reinforce one another. Certainly, one would not expect the charter sector to become increasingly 


selective if the quality of educational services were declining. 


The more rapid rate of progress by student cohorts in the charter than in the district sector 
may help to explain rising opposition to charters by school districts, district superintendents, and 
district teacher organizations. When a sector of the economy begins to lose market share, those 


working within the sector typically seek government action to curb losses. As the automobile 


28 


industry in the United States lost market share to rapidly improving companies abroad, demands 
for tariffs and trade restrictions escalated. Much the same may be happening in contemporary 


school politics. 


The relative rate of improvement at charters as compared to district schools is greatest for 
cohorts of African American students. Our results are thus consistent with experimental studies, 
which have generally found strong positive effects on African American students 
(Abdulkadiroglu et al., 2011; Cohodes et al, 2013; Curto & Fryer, 2014; Dobbie & Fryer, 2013; 


Hoxby, Muraka, & Kang, 2009; Winters, 2020). 


We also find gains over time for cohorts of White American students. Although the gains 
are less than those for Black cohorts, the finding points in a different direction than results from 
prior research, which generally find neutral to negative charter effects on White students 
(CREDO, 2013; Gleason et al., 2010). We see no relative improvement in cohort progress in the 


charter sector for Hispanic Americans and Asian Americans. 


Cohort gains at charter schools relative to district schools is substantially higher for 
students from the lowest SES quartile of the distribution than for those from higher SES 
quartiles. This finding is generally consistent with research on charter effects, which for the most 
part report more positive results for disadvantaged students than for those from higher SES 


backgrounds (CREDO; 2015; Walters, 2018). 


Charters enjoy the largest relative advantage among students attending schools in the 
Northeast, a finding consistent with a large number of experimental studies that find positive 
charter impacts on students in New York City, Boston, Newark, and Washington, D.C. 


(Abdulkadiroglu et al., 2011; Cohodes et al, 2013; Curto & Fryer, 2014; Dobbie & Fryer, 2013; 
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Hoxby, Muraka, & Kang, 2009; Winters, 2020). We find a smaller but still positive relative 
advantage in the rate of improvement in the Midwest and South but no difference in 
improvement rates between the two sectors in the West. The last finding is consistent with the 
quite mixed set of findings from studies of charters in the mountain and Pacific Coast states 


(CREDO, 2009; 2014). 


The regional variation in improvement gains is difficult to reconcile with the slow growth 
of charter schools in the Northeast and the continued quite rapid growth in the mountain states of 
the West (Cheng & Peterson, 2017). It appears that charters are expanding the most where they 
have the least to offer, and vice versa. The irony may once again be explained by the differential 
threat charters pose to district schools in various parts of the country. Where charters are 
improving at a more rapid rate than district schools, the districts are more likely to mobilize their 
considerable political resources in opposition. But where charters are improving at a rate that 
poses less of a threat to the district sector, charter expansion is ignored by the district sector, 


especially if student enrollment is rising more generally. 


Sixty-five percent of charter students are enrolled in schools located within big cities. 
Here we find steady gains by students in charters relative to district schools. Meanwhile, 
differences between charter and district schools in suburbia are smaller and inconsistent, 
depending on grade level. Those findings resemble results from prior research, which generally 
finds more positive charter impacts in urban than in suburban areas (Abdulkadiroglu et al., 2017; 


Angrist et al., 2010, 2012; 2013). 


Given the rising achievement levels at charter schools, it is unlikely that the slowdown in 
the sector’s growth rate is due to declining productivity. It is more likely that political resistance 


to charters is increasing as both the management and labor sides of the district sector become 
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increasingly concerned that charters might prove to be as disruptive an innovation as the 


transistor. 
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Figure 1. Estimated changes in average student performance on NAEP by school sector and subject, grade 8: Years 
2005-17 

Note. Estimated changes in student performance are differences in standard deviations (sd) between mean 
performance level in year test was administered and initial test year, which is set at zero. The lines represent a 
quadratic fit. Magnitude of the estimated change in average performance is displayed in parentheses at end of each 
line. Estimated mean performance level for each year is indicated by notation displayed in legend. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2005—2017 Main NAEP. 
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Table 1. Share of enrollment in charter sector in NAEP and NCES 


Ethnicity Survey Grade 2001 2005 2006 2007 2009 2011 2013 2015 2017 2018 


All NCES Zl 3.6 4.6 5.4 6 
NAEP 4 1.6 2e1 2.4 2.9 4 4.8 3.9 
8 1.5 1.8 pi 2.0 3.6 53 5.6 
Black NCES 33,2 28.9 25.8 
NAEP 4 3339 38 38.6 31.2 34.9 Ziad 31.9 
8 36.7 36.6 28 31.9 33.3 27.2 28.6 
White NCES 42.7 36.2 321 
NAEP 4 42 38.7 Boal 36.3 30.9 Bil 34.4 
8 38.9 34.7 34.1 31.2 30.4 29 30.3 
Hispanic NCES 19.4 2h 33.1 
NAEP 4 19.1 17.8 2 21.1 26.3 28.6 25.6 
8 18.7 22:5 31.5 SL 29.1 34.7 S21 
Asian NCES 2.9 Set 4.4 
NAEP 4 a2 2 ool 32 4.2 4.3 4.4 
8 2.8 3.7 auf 3.9 4.7 Die 6 


Note: The table displays share of enrollment in percentage in charter sector as a proportion of public sector in NAEP and NCES. NAEP estimates use survey 
weights. NCES data combine elementary and secondary grades. 
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Sources: U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), “Public Elementary/Secondary School Universe 
Survey,” 2000-01 and 2017-18. See Digest of Education Statistics 2019, table 216.30., See Digest of Education Statistics 2014, 2015, 2016, 2017, 2018, and 
2019, table 216.20., and 2005-17 Main NAEP. 
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Table 2. NAEP scale scores by school sector 


2005 2017 
Grade Subject _ District Charter Difference (sd) District Charter__Difference _(sd) 
4 Math 237 232 5* 0.15 239 236 3 0.09 
(29) (29) (29) (32) (32) (32) 
Reading 217 216 2 0.06 221 222 -1 -0.03 
(36) (36) (36) (38) (37) (38) 
8 Math 278 268 10** 0.28 282 282 0 0 
(36) (37) (36) (39) (39) (39) 
Reading 260 255 oF 0.14 265 266 -1 -0.03 
(35) (36) (35) (36) (35) (36) 


Note: Difference displays the district sector advantage in achievement over charter sector. Standard deviations are in 
parentheses. Columns headed by sd display the district sector advantage in standard deviations. It was estimated by 
dividing the district sector advantage by the average standard deviation of sectors and subjects for each year and 
grade. *p <0.05, **p<0.01, ***p<0.001. 
Source: U.S. Department of Education, National Center for Education Statistics, 2005-2017 Main NAEP. 
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Table 3. Estimated changes in average student performance on NAEP, 2005-17, by sector (in 
standard deviations) 


Grade Subject District _ Charter Difference District | Charter Difference 


4 Math 0.076 0.172 0.096 0.169 0.214 0.045 
(0.003) (0.019) (0.003) (0.018) 
Reading 0.084 0.209 0.125 0.189 0.262 0.073 
(0.003) (0.018) (0.003) (0.018) 
8 Math 0.101 0.344 0.243 0.218 0.415 0.197 
(0.003) (0.018) (0.003) (0.019) 
Reading 0.145 0.254 0.109 0.267 O327 0.060 
(0.003) (0.018) (0.003) (0.018) 
Average 0.102 0.245 0.143 0.211 0.305 0.094 
Controls No No No Yes Yes Yes 


Note: The table displays mean cohort change in student performance in standard deviations (s.d.) by subject, grade, 
and sector between 2005 and 2017. The s.d. is the estimated change over the period 2005-17. Estimates are obtained 
from a quadratic fit. Standard errors are in parentheses. The two rightmost columns report the difference between 
changes in achievement in charter and district sectors. Models with controls include dummies for gender, eligiblity 
for free and reduced-price lunch, and for those in 8th grade, the socioeconomic status of student background. 
Source: U.S. Department of Education, National Center for Education Statistics, 2005-17 Main NAEP. 
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Table 4. Estimated changes in average student performance on NAEP, 2005-17, by sector and 


ethnicity (in standard deviations) 


Grade Subject District Charter Difference District Charter _ Difference 
African American 
4 Math 0.12 0.265 0.145 0.179 0.317 0.138 
(0.007) (0.030) (0.007) (0.030) 
Reading 0.156 0.277 0.121 0.212 0.328 0.116 
(0.007) (0.030) (0.007) (0.030) 
8 Math 0.113 0.462 0.349 O22 0.518 0.298 
(0.007) (0.029) (0.008) (0.030) 
Reading 0.177 0.327 0.150 0.298 0.423 0.125 
(0.007) (0.027) (0.007) (0.028) 
Average _ 0.142 0.333 0.191 0.227 0.397 0.169 
White American 
4 Math 0.076 0.082 0.006 0.164 0.144 -0.020 
(0.004) (0.032) (0.004) (0.033) 
Reading 0.099 0.187 0.088 0.184 0.236 0.052 
(0.004) (0.031) (0.004) (0.032) 
8 Math 0.1 0.375 0.275 0.161 0.353 0.192 
(0.004) (0.033) (0.003) (0.036) 
Reading 0.139 0.242 0.103 0.208 0.237 0.029 
(0.004) (0.032) (0.004) (0.034) 
Average 0.104 0.222 0.118 0.179 0.243 0.063 
Controls — No No No Yes Yes Yes 
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Table 4 (Cont’d). Estimated changes in average student performance on the NAEP, 2005-17, by 
sector and ethnicity (in standard deviations) 


Grade Subject District Charter Difference District Charter _ Difference 


Hispanic American 


4 Math 0.173 0.195 0.022 0.193 0.236 0.043 
(0.007) (0.047) (0.007) (0.047) 
Reading 0.166 O22) 0.061 0.196 0.296 0.100 
(0.007) (0.042) (0.007) (0.041) 
8 Math 0.225 0.171 -0.054 0.264 0.203 -0.061 
(0.007) (0.042) (0.008) (0.046) 
Reading 0.291 0.264 -0.027 0.332 0.267 -0.065 
(0.007) (0.042) (0.008) (0.045) 
Average _ 0.214 0.214 0.001 0.246 0.251 0.004 
Asian American 
4 Math 0.238 0.305 0.067 0.268 0.397 0.129 
(0.013) (0.099) (0.013) (0.102) 
Reading 0.276 0.224 -0.052 0.319 0.265 -0.054 
(0.013) (0.107) (0.012) (0.107) 
8 Math 0.398 0.476 0.078 0.451 0.521 0.070 
(0.014) (0.090) (0.014) (0.084) 
Reading 0.352 0.324 -0.028 0.423 0.262 -0.161 
(0.014) (0.089) (0.014) (0.089) 
Average _0.316 0.332 0.016 0.365 0.361 -0.004 
Controls No No No Yes Yes Yes 


Note and Source: See Table 3. 
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Table 5. Estimated changes in student performance on NAEP, 2005-17, by sector and SES, 


grade 8 
SES quartile Subject District Charter Difference District Charter Difference 
Top Math 0.017 0.274 0.257 0.045 0.212 0.167 
(0.006) (0.038) (0.006) (0.036) 
Reading 0.092 0.212 0.120 0.132 0.220 0.088 
(0.006) (0.037) (0.006) (0.036) 
Average 0.055 0.243 0.189 0.089 0.216 0.128 
Second Math 0.007 0.338 0.331 0.043 0.273 0.230 
(0.007) (0.045) (0.007) (0.043) 
Reading 0.079 0.165 0.086 0.106 0.139 0.033 
(0.007) (0.045) (0.007) (0.044) 
Average 0.043 0252 0.209 0.075 0.206 0.132 
Third Math 0.073 0.282 0.209 0.087 0.286 0.199 
(0.009) (0.057) (0.008) (0.053) 
Reading 0.108 0.166 0.058 0.130 0.180 0.050 
(0.009) (0.053) (0.008) (0.049) 
Average 0.091 0.224 0.134 0.109 0.233 0.125 
Bottom Math 0.212 0.483 0.271 0.211 0.490 0.279 
(0.005) (0.034) (0.005) (0.032) 
Reading 0.245 0.310 0.065 0.238 0.311 0.073 
(0.005) (0.032) (0.005) (0.030) 
Average (0.229 0.397 0.168 0.225 0.401 0.176 
Controls No No No Yes Yes Yes 


Note and Source: See Table 3. 
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Table 6. Estimated changes in average student performance on NAEP, 2005-17, by sector and 
free and reduced-price lunch (FRL) status (in standard deviations) 


Grade Subject District Charter Difference 
High income (Ineligible for FRL) 
4 Math 0.231 0.231 0.000 
(0.004) (0.029) 
Reading 0.217 0.281 0.064 
(0.004) (0.028) 
8 Math 0.246 0.529 0.283 
(0.004) (0.029) 
Reading 0.259 0.345 0.086 
(0.004) (0.028) 
Average 0.238 0.347 0.108 
Low income (Eligible for FRL) 
4 Math 0.123 0.26 0.137 
(0.004) (0.026) 
Reading 0.153 0.268 0.115 
(0.004) (0.026) 
8 Math 0.157 0.35 0.193 
(0.004) (0.025) 
Reading W215 0.358 0.143 
(0.004) (0.025) 
Average 0.162 0.309 0.147 


Note and Source: See Table 3. 
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Table 7. Estimated changes in average student performance on NAEP, 2005-17, by region and 
sector (in standard deviations) 


Region Grade _Subject__ District Charter Difference District Charter _ Difference 
Northeast 4 Math -0.039 0.173 0.212 0.037 0.289 0.252 
(0.007) (0.057) (0.006) (0.055) 
Reading 0.048 0.087 0.039 0.129 0.263 0.134 
(0.007) (0.064) (0.006) (0.062) 
8 Math 0.104 0.423 0.319 0.172 0.566 0.394 
(0.007) (0.062) (0.007) (0.065) 
Reading 0.089 0.084 -0.005 0.195 0.389 0.194 
(0.007) (0.067) (0.007) (0.072) 
Average 0.051 0.192 0.141 0.133 0377 0.244 
Midwest 4 Math 0.068 0.177 0.109 0.149 0.147 -0.002 
(0.006) (0.041) (0.005) (0.038) 
Reading 0.047 0.202 0.155 0.14 O19) 0.051 
(0.006) (0.040) (0.006) (0.037) 
8 Math 0.096 0.412 0.316 0.172 0.495 0.323 
(0.006) (0.046) (0.006) (0.044) 
Reading 0.11 0.225 0.115 0.196 0.278 0.082 
(0.006) (0.041) (0.006) (0.039) 
Average 0.080 0.254 0.174 0.164 0.278 0.114 
South 4 Math U.E25 0.125 0.000 0.195 0.217 0.022 
(0.005) (0.033) (0.004) (0.030) 
Reading 0.088 0.188 0.100 0.186 0.273 0.087 
(0.005) (0.032) (0.004) (0.030) 
8 Math 0.071 0.278 0.207 0.153 0.256 0.103 
(0.005) (0.026) (0.005) (0.025) 
Reading 0.128 0.172 0.044 0.215 0.264 0.049 
(0.005) (0.025) (0.005) (0.024) 
Average 0.103 0.191 0.088 0.187 0.253 0.065 
West 4 Math 0.091 0.082 -0.009 0.194 0.153 -0.041 
(0.006) (0.036) (0.005) (0.035) 
Reading 0.146 0.183 0.037 0.246 0.272 0.026 
(0.006) (0.033) (0.005) (0.033) 
8 Math 0.168 0.218 0.050 0.278 0.258 -0.020 
(0.006) (0.035) (0.006) (0.037) 
Reading 0.262 0.294 0.032 0.38 0.302 -0.078 
(0.006) (0.034) (0.006) (0.036) 
Average 0.167 0.194 0.028 0.275 0.246 -0.028 
Controls No No No Yes Yes Yes 


Note and Source: See Table 3. 
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Table 8. Estimated changes in average student performance on NAEP, 2007-17, by sector and 
locale (in standard deviations) 


Locale Grade Subject __ District Charter _ Difference District Charter _ Difference 
City 4 Math 0.03 0.258 0.228 0.075 0.221 0.146 
(0.005) (0.025) (0.005) (0.024) 
Reading 0.059 0.342 0.283 0.121 0.318 O97 
(0.005) (0.023) (0.005) (0.022) 
8 Math 0.081 0.17 0.089 0.132 0.163 0.031 
(0.006) (0.021) (0.005) (0.021) 
Reading 0.165 0.167 0.002 0.23 0.158 -0.072 
(0.005) (0.022) (0.005) (0.021) 
Average 0.084 0.234 0.151 0.140 0.215 0.076 
Suburb 4 Math 0.014 = -0.011 -0.025 0.117 0.019 -0.098 
(0.006) (0.038) (0.005) (0.036) 
Reading 0.054 0.082 0.028 0.169 0.104 -0.065 
(0.006) (0.035) (0.005) (0.034) 
8 Math 0.042 0.213 0.171 0.129 = 0.223 0.094 
(0.006) (0.040) (0.006) (0.038) 
Reading 0.135 0.289 0.154 0.23 0.296 0.066 
(0.006) (0.045) (0.005) (0.044) 
Average 0.061 0.143 0.082 0.161 0.161 -0.001 
Town & rural 4 Math 0.001 0.075 0.074 0.102 0.022 -0.080 
(0.005) (0.052) (0.005) (0.049) 
Reading -0.015 0.168 0.183 0.09 0.201 0.111 
(0.005) (0.050) (0.005) (0.049) 
8 Math -0.045 0.267 0.312 0.05 0.327 0.277 
(0.005) (0.044) (0.005) (0.043) 
Reading 0.029 0.176 0.147 0.131 0.24 0.109 
(0.005) (0.044) (0.005) (0.043) 
Average -0.008 0.172 O19 0.093 0.198 0.104 
Controls No No No Yes Yes Yes 


Note and Source: See Table 3. 
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Appendix Table A. Unweighted observations by school sector on NAEP, 2005-17 


Category 
Levels 


Gender 
Male 
Female 
Male 
Female 

Ethnicity 
White 
Hispanic 
Black 
Asian 
White 
Hispanic 
Black 
Asian 

SES quartiles 
Top 
Second 
Third 
Bottom 

FRL 
Ineligible 
Free & reduced 
Ineligible 
Free & reduced 


Grade 
4 
8 


4 


8 


District 
Math 
1,162,970 
1,038,770 


570,770 
592,200 
511,790 
526,980 


638,100 
192,710 
220,240 
56,560 

291,290 
173,430 
178,330 
51,570 


221,150 
219,550 
221,470 
220,620 


542,040 
595,120 
535,120 
483,660 


Reading 
1,165,740 
1,034,670 


575,310 
590,430 
512,100 
522,570 


644,890 
193,290 
214,650 
57,140 

592,610 
172,030 
175,150 
50,750 


6,860 
8,450 
6,540 
7,390 


548,760 
591,400 
535,690 
479,110 


Charter 
Math 
33,420 
34,930 


16,910 
16,500 
17,710 
17,220 


10,680 
13200 
6,980 
1,480 
9,590 
14,010 
8,520 
1,840 


220,800 
219,400 
221,340 
220,340 


12,960 
19,180 
12,830 
20,390 


Reading 
33,790 
34,630 


17,050 
16,740 
17,770 
16,870 


10,920 
13,340 
7,030 
1,490 
9,500 
13,810 
8,470 
1,860 


6,920 
8,320 
6,380 
7,370 


13,300 
19,180 
12,880 
19,990 


Appendix Table A (Cont'd). Unweighted observations by school sector on NAEP, 2005-17 


District Charter 
Category Grade Math Reading Math Reading 
Region 
Northeast 4 192,650 193,770 3,030 3,060 
Midwest 254,740 256,650 6,380 6,420 
South 408,240 405,140 13,680 13,810 
West 294,700 297,580 10,330 10,490 
Northeast 8 174,110 173,940 3,130 3,080 
Midwest 234,170 233,310 5,100 5,100 
South 359,810 356,960 15,690 15,500 
West 261,290 261,220 11,010 10,960 
Locale 
City 4 339,450 339,810 19,430 19,630 
Suburb 278,190 280,970 6,890 7,030 
Town & rural 373,940 379,710 4,720 4,840 
City 8 286,670 285,050 21,830 21,550 
Suburb 240,780 240,460 6,040 6,050 
Town & rural 352,690 352,820 4,680 4,690 


Note: Information on locale is available since 2007. Information on SES is derived from student-reported parents’ 
education level, number of books, and computer at home. 
Source: See Table 3. 
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